Image sensor with in-pixel depth sensing

ABSTRACT

An imaging area in an image sensor includes a plurality of photo detectors. A light shield is disposed over a portion of two photo detectors to partially block light incident on the two photo detectors. The two photo detectors and the light shield combine to form an asymmetrical pixel pair. The two photo detectors in the asymmetrical pixel pair can be two adjacent photo detectors. The light shield can be disposed over contiguous portions of the two adjacent photo detectors. A color filter array can be disposed over the plurality of photo detectors. The filter elements disposed over the two photo detectors can filter light representing the same color or different colors.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/198,540, filed Mar. 5, 2014, titled “Image Sensor with In-Pixel DepthSensing,” which claims the benefit under 37 C.F.R. §119(e) of U.S.Provisional Patent Application No. 61/785,920, filed on Mar. 14, 2013,entitled “Image Sensor With In-Pixel Depth Sensing,” which is herebyincorporated by reference in its entirety.

TECHNICAL FIELD

The present invention relates generally to electronic devices, and morespecifically, to image sensors for electronic devices.

BACKGROUND

Cameras and other image recording devices often use one or more imagesensors, such as a charge-coupled device (CCD) image sensor or acomplementary metal-oxide-semiconductor (CMOS) image sensor. When animage of a scene is captured, the scene can include objects that can bepositioned or illuminated in a way that can make it difficult torepresent the objects with acceptable detail. For example, an object canbe positioned in a shadow, or two objects can be sufficiently far apartin distance that the camera is not able to focus on both objectsadequately. This means one or both objects may be blurred in the image.

Depth of field is a term that can represent the distance between thenearest and farthest objects in a scene that appear acceptably sharp inan image. Typically, a camera may focus at only one distance, so thesharpness of objects located at different distances in the scenedecrease on either side of the focused distance. If a camera is focusedon an object in a scene that is near the camera, objects located fartheraway from the camera, or closer to the camera than the object, can beblurred in the image. Likewise, if a camera is focused on an object thatis farther away from the camera, objects that are closer to the cameracan be blurred in the image.

SUMMARY

Embodiments described herein may relate to or take the form of animaging area in an image sensor includes a plurality of photo detectors.A light shield is disposed over a portion of two photo detectors to onlypartially block light incident on the two photo detectors. By way ofexample only, the portion of the two photo detectors covered by thelight shield can be half of each of the two photo detectors. The twophoto detectors can be two adjacent photo detectors, and the lightshield can be disposed over contiguous portions of the two adjacentphoto detectors.

Other embodiments may include a color filter array disposed over theplurality of photo detectors. The color filter array includes aplurality of filter elements and the filter elements disposed over thetwo photo detectors can filter light wavelengths representing the samecolor or different colors.

Further embodiments may include an imaging area in an image sensor whichitself includes a plurality of pixels, where at least one pixel may bedivided into two or more sub-pixel regions. Each sub-pixel can include aphoto detector and a transfer transistor connected between the photodetector and a common node. A portion of the sub-pixels in the imagingarea can be configured as asymmetrical photo detector pairs, where eachasymmetrical photo detector pair includes two photo detectors and alight shield disposed over portions of the two photo detectors topartially block light received by each of the two photo detectors in theasymmetrical photo detector pair.

Other embodiments described herein may include or take the form of amethod for determining a focus setting for an image capture device thatmay include includes an image sensor having an asymmetrical photodetector pair. The method can include the steps of capturing one or moreimages using a first focus setting and analyzing a first signal responseoutput from each photo detector in the asymmetrical photo detector pair.A determination can be made as to whether a difference between the firstsignal responses is equal to or less than a threshold value. If thedifference is less than or equal to the threshold value, an initialfocus setting is set to the first focal setting. If the difference isgreater than the threshold value, the method can repeat using differentfocus settings until a focus setting produces a difference in signalresponses that equals or is less than the threshold value.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are better understood with reference to thefollowing drawings. The elements of the drawings are not necessarily toscale relative to each other. Identical reference numerals have beenused, where possible, to designate identical features that are common tothe figures.

FIG. 1A depicts a front perspective view of an example electronic deviceincluding one or more cameras.

FIG. 1B depicts a rear perspective view of the example electronic deviceof FIG. 1A.

FIG. 2 depicts a simplified block diagram of the example electronicdevice of FIG. 1.

FIG. 3 depicts a cross-section view of the example electronic device ofFIG. 1A taken along line 3-3 in FIG. 1A.

FIG. 4 depicts a simplified block diagram of one example of an imagesensor that may be suitable for use as image sensor.

FIG. 5 depicts a simplified schematic view of a pixel that may besuitable for use in an image sensor.

FIG. 6 depicts a simplified schematic view of one example of a sub-pixelarrangement that may be suitable for use in an image sensor.

FIG. 7 depicts an example color filter array that may be suitable foruse with an image sensor having pixels configured as shown in FIG. 5.

FIG. 8 depicts a Bayer color filter array pattern.

FIG. 9 depicts a color filter array that may be suitable for use with animage sensor having sub-pixels configured as shown in FIG. 6.

FIG. 10 depicts a top view of an example asymmetrical photo detectorpair.

FIG. 11 depicts a simplified cross-sectional view of the exampleasymmetrical photo detector pair shown in FIG. 10.

FIG. 12 depicts an example of angular signal responses of theasymmetrical photo detector pair shown in FIG. 11.

FIG. 13 depicts a simplified top view of an example image sensor showingone arrangement of asymmetrical photo detector pairs.

FIG. 14 depicts a simplified top view of an example image sensor showinganother arrangement of asymmetrical photo detector pairs.

FIG. 15 depicts an example flowchart of a method for determining aninitial focus setting for an image capture device.

DETAILED DESCRIPTION

Embodiments described herein include an image sensor that is configuredto provide depth of field information. The depth of field informationcan be used, for example, to determine an initial lens focus setting, tobetter focus one or more lenses when an image is captured, for gesturerecognition, and in a three dimensional (3D) application. In oneembodiment, an image sensor can operate in three different modes. Onemode is a charge summing mode, another mode is a high dynamic rangemode, and the third mode is a depth of field mode.

The pixels in an image sensor can be implemented as sub-pixels. Forexample, a pixel can be divided into four sub-pixels, where eachsub-pixel can include a photo detector and a transfer transistor. Thephoto detectors accumulate charge in response to incident light. Thetransfer transistors in the four sub-pixels can be connected to a commonsense region that is shared by the four sub-pixels. The gate of eachtransfer transistor can be independently pulsed to transfer accumulatedcharge from one or more photo detectors to the shared sense region. Theaccumulated charge in the one or more photo detectors can be transferredsimultaneously, sequentially, or in groups. The charge on the sharedsense region can then be read out.

In some embodiments, a color filter array can be disposed over an imagesensor. The color filter array may be a mosaic of filter elements, whereeach filter element may be typically disposed over a pixel. Each filterelement can restrict the wavelengths of light that strike a pixel. Thelight wavelengths can be restricted by color. For example, one filterelement can transmit light wavelengths associated with the color red,another color filter element can transmit light wavelengths associatedwith the color green, and another color filter element can transmitlight wavelengths associated with the color blue. When a pixel isconfigured into sub-pixels, the same filter element can be disposed overthe sub-pixels in a pixel, allowing the sub-pixels to detect the samewavelengths of light (or color). The accumulated charge in two or moresub-pixels can be summed together by selectively pulsing the gates ofthe transfer transistors in the two or more sub-pixels. Thus,accumulated charge representing the same color can be summed together.Charge summing can, for example, improve the signal-to-noise ratio (SNR)of an image.

The dynamic range of an image sensor quantifies the ability of the imagesensor to adequately image both high light areas in a scene and low darkareas or shadows in the scene. In general, the dynamic range of an imagesensor may be less than that of the human eye. The limited dynamic rangeof an image sensor can result in an image losing details in the brighterareas or in the darker areas of the scene. A variety of algorithms havebeen produced to improve the dynamic range of image sensors. One suchalgorithm varies the integration times (the time light is collected) ofthe sub-pixels in the image sensor, which produces multiple images of ascene. For example, some sub-pixels can have a shorter integration timewhile other sub-pixels can have a longer integration time. Thesub-pixels with the shorter integration time can better capture thebrighter areas in a scene and the sub-pixels with the longer integrationtime can better capture darker areas in the scene. The image signalsfrom sub-pixels having shorter and longer integration times can then becombined to produce a final high dynamic range image that has moredetail in the lighter and in the darker areas of the image.

As described earlier, the embodiments described herein can provide animage sensor or image capture device that may be capable of operating inthree different modes; a charge summing mode, a high dynamic range mode,and a depth of field information mode. The depth of field informationcan be obtained using one or more asymmetrical photo detector pairsincluded in an imaging area of an image sensor. Each asymmetrical photodetector pair includes two photo detectors and a light shield disposedover a portion of the two photo detectors to only partially block lightthat is incident on the two photo detectors. The signal responsesproduced by the two photo detectors in one or more asymmetrical photodetector pairs can be analyzed to determine a focus setting for an imagecapture device. A color filter array can be disposed over the imagingarea. The filter elements disposed over the two photo detectors in eachasymmetrical photo detector pair can filter light representing the samecolor or different colors.

Directional terminology, such as “top”, “bottom”, “front”, “back”,“leading”, “trailing”, etc., is used with reference to the orientationof the Figure(s) being described. Because components in variousembodiments can be positioned in a number of different orientations, thedirectional terminology is used for purposes of illustration only and isin no way limiting. When used in conjunction with layers of an imagesensor wafer, image sensor die, or corresponding image sensor, thedirectional terminology is intended to be construed broadly, andtherefore should not be interpreted to preclude the presence of one ormore intervening layers or other intervening image sensor features orelements. Thus, a given layer that is described herein as being formedon, formed over, disposed on, or disposed over another layer may beseparated from the latter layer by one or more additional layers.

Referring now to FIGS. 1A-1B, there are shown front and rear perspectiveviews of an electronic device that includes one or more cameras. Theelectronic device 100 includes a first camera 102, a second camera 104,an enclosure 106, a display 110, an input/output (I/O) member 108, and aflash 112 or light source for the camera or cameras. The electronicdevice 100 can also include one or more internal components (not shown)typical of a computing or electronic device, such as, for example, oneor more processors, memory components, network interfaces, and so on.

In the illustrated embodiment, the electronic device 100 is implementedas a smart telephone. Other embodiments, however, are not limited tothis construction. Other types of computing or electronic devices caninclude one or more cameras, including, but not limited to, a netbook orlaptop computer, a tablet computer, a digital camera, a printer, ascanner, a video recorder, and a copier.

As shown in FIGS. 1A-1B, the enclosure 106 can form an outer surface orpartial outer surface and protective case for the internal components ofthe electronic device 100, and may at least partially surround thedisplay 110. The enclosure 106 can be formed of one or more componentsoperably connected together, such as a front piece and a back piece.Alternatively, the enclosure 106 can be formed of a single pieceoperably connected to the display 110.

The I/O member 108 can be implemented with any type of input or outputmember. By way of example only, the I/O member 108 can be a switch, abutton, a capacitive sensor, or other input mechanism. The I/O member108 allows a user to interact with the electronic device 100. Forexample, the I/O member 108 may be a button or switch to alter thevolume, return to a home screen, and the like. The electronic device caninclude one or more input members or output members, and each member canhave a single I/O function or multiple I/O functions.

The display 110 can be operably or communicatively connected to theelectronic device 100. The display 110 can be implemented with any typeof suitable display, such as a high resolution display or an activematrix color liquid crystal display. The display 110 can provide avisual output for the electronic device 100 or function to receive userinputs to the electronic device. For example, the display 110 can be amulti-touch capacitive sensing touchscreen that can detect one or moreuser inputs.

The electronic device 100 can also include a number of internalcomponents. FIG. 2 illustrates one example of a simplified block diagramof the electronic device 100. The electronic device can include one ormore processors 200, storage or memory components 202, input/outputinterface 204, power sources 206, and sensors 208, each of which will bediscussed in turn below.

The one or more processors 200 can control some or all of the operationsof the electronic device 100. The processor(s) 200 can communicate,either directly or indirectly, with substantially all of the componentsof the electronic device 100. For example, one or more system buses 210or other communication mechanisms can provide communication between theprocessor(s) 200, the cameras 102, 104, the display 110, the I/O member108, or the sensors 208. The processor(s) 200 can be implemented as anyelectronic device capable of processing, receiving, or transmitting dataor instructions. For example, the one or more processors 200 can be amicroprocessor, a central processing unit (CPU), an application-specificintegrated circuit (ASIC), a digital signal processor (DSP), orcombinations of multiple such devices. As described herein, the term“processor” is meant to encompass a single processor or processing unit,multiple processors, multiple processing units, or other suitablyconfigured computing element or elements.

The memory 202 can store electronic data that can be used by theelectronic device 100. For example, the memory 202 can store electricaldata or content such as, for example, audio files, document files,timing signals, and image data. The memory 202 can be configured as anytype of memory. By way of example only, memory 202 can be implemented asrandom access memory, read-only memory, Flash memory, removable memory,or other types of storage elements, in any combination.

The input/output interface 204 can receive data from a user or one ormore other electronic devices. Additionally, the input/output interface204 can facilitate transmission of data to a user or to other electronicdevices. For example, in embodiments where the electronic device 100 isa smart telephone, the input/output interface 204 can receive data froma network or send and transmit electronic signals via a wireless orwired connection. Examples of wireless and wired connections include,but are not limited to, cellular, WiFi, Bluetooth, and Ethernet. In oneor more embodiments, the input/output interface 204 supports multiplenetwork or communication mechanisms. For example, the input/outputinterface 204 can pair with another device over a Bluetooth network totransfer signals to the other device while simultaneously receivingsignals from a WiFi or other wired or wireless connection.

The power source 206 can be implemented with any device capable ofproviding energy to the electronic device 100. For example, the powersource 206 can be a battery or a connection cable that connects theelectronic device 100 to another power source such as a wall outlet.

The sensors 208 can by implemented with any type of sensors. Examples ofsensors include, but are not limited to, audio sensors (e.g.,microphones), light sensors (e.g., ambient light sensors), gyroscopes,and accelerometers. The sensors 208 can be used to provide data to theprocessor 200, which may be used to enhance or vary functions of theelectronic device.

As described with reference to FIGS. 1A and 1B, the electronic device100 includes one or more cameras 102, 104 and optionally a flash 112 orlight source for the camera or cameras. FIG. 3 is a simplifiedcross-section view of the camera 102 taken along line 3-3 in FIG. 1A.Although FIG. 3 illustrates the first camera 102, those skilled in theart will recognize that the second camera 104 can be substantiallysimilar to the first camera 102. In some embodiments, one camera mayinclude a global shutter configured image sensor and one camera caninclude a rolling shutter configured image sensor. In other examples,one camera can include an image sensor with a higher resolution than theimage sensor in the other camera.

The cameras 102, 104 include an imaging stage 300 that is in opticalcommunication with an image sensor 302. The imaging stage 300 isoperably connected to the enclosure 106 and positioned in front of theimage sensor 302. The imaging stage 300 can include conventionalelements such as a lens, a filter, an iris, and a shutter. The imagingstage 300 directs, focuses or transmits light 304 within its field ofview onto the image sensor 302. The image sensor 302 captures one ormore images of a subject scene by converting the incident light intoelectrical signals.

The image sensor 302 is supported by a support structure 306. Thesupport structure 306 can be a semiconductor-based material including,but not limited to, silicon, silicon-on-insulator (SOI) technology,silicon-on-sapphire (SOS) technology, doped and undoped semiconductors,epitaxial layers formed on a semiconductor substrate, well regions orburied layers formed in a semiconductor substrate, and othersemiconductor structures.

Various elements of imaging stage 300 or image sensor 302 can becontrolled by timing signals or other signals supplied from a processoror memory, such as processor 200 in FIG. 2. Some or all of the elementsin the imaging stage 300 can be integrated into a single component.Additionally, some or all of the elements in the imaging stage 300 canbe integrated with image sensor 302, and possibly one or more additionalelements of electronic device 100, to form a camera module. For example,a processor or a memory may be integrated with the image sensor 302 inembodiments.

Referring now to FIG. 4, there is shown a top view of one example of animage sensor suitable for use as image sensor 302. The image sensor 400can include an image processor 402 and an imaging area 404. The imagingarea 404 is implemented as a pixel array that includes pixels 406. Inthe illustrated embodiment, the pixel array is configured in a row andcolumn arrangement. However, other embodiments are not limited to thisconfiguration. The pixels in a pixel array can be arranged in anysuitable configuration, such as, for example, a hexagon configuration.

The imaging area 404 may be in communication with a column select 408through one or more column select lines 410 and a row select 412 throughone or more row select lines 414. The row select 412 selectivelyactivates a particular pixel 406 or group of pixels, such as all of thepixels 406 in a certain row. The column select 408 selectively receivesthe data output from the select pixels 406 or groups of pixels (e.g.,all of the pixels with a particular column).

The row select 412 and/or the column select 408 may be in communicationwith an image processor 402. The image processor 402 can process datafrom the pixels 406 and provide that data to the processor 200 and/orother components of the electronic device 100. It should be noted thatin some embodiments, the image processor 402 can be incorporated intothe processor 200 or separate therefrom.

Referring now to FIG. 5, there is shown a simplified schematic view of apixel that is suitable for use as pixels 406. The pixel 500 includes aphoto detector 502, a transfer transistor 504, a sense region 506, areset (RST) transistor 508, a readout transistor 510, and a row select(RS) transistor 512. The sense region 506 is represented as a capacitorin the illustrated embodiment because the sense region 506 cantemporarily store charge received from the photo detector 502. Asdescribed below, after charge is transferred from the photo detector502, the charge can be stored in the sense region 506 until the gate ofthe row select transistor 512 is pulsed.

One terminal of the transfer transistor 504 is connected to the photodetector 502 while the other terminal is connected to the sense region506. One terminal of the reset transistor 508 and one terminal of thereadout transistor 510 are connected to a supply voltage (Vdd) 514. Theother terminal of the reset transistor 508 is connected to the senseregion 506, while the other terminal of the readout transistor 510 isconnected to a terminal of the row select transistor 512. The otherterminal of the row select transistor 512 is connected to an output line410.

By way of example only, in one embodiment the photo detector 502 isimplemented as a photodiode (PD) or pinned photodiode, the sense region506 as a floating diffusion (FD), and the readout transistor 510 as asource follower transistor (SF). The photo detector 502 can be anelectron-based photodiode or a hole based photodiode. It should be notedthat the term photo detector as used herein is meant to encompasssubstantially any type of photon or light detecting component, such as aphotodiode, pinned photodiode, photogate, or other photon sensitiveregion. Additionally, the term sense region as used herein is meant toencompass substantially any type of charge storing or charge convertingregion.

Those skilled in the art will recognize that the pixel 500 can beimplemented with additional or different components in otherembodiments. For example, a row select transistor can be omitted and apulsed power supply mode used to select the pixel, the sense region canbe shared by multiple photo detectors and transfer transistors, or thereset and readout transistors can be shared by multiple photo detectors,transfer gates, and sense regions.

When an image is to be captured, an integration period for the pixelbegins and the photo detector 502 accumulates photo-generated charge inresponse to incident light. When the integration period ends, theaccumulated charge in the photo detector 502 is transferred to the senseregion 506 by selectively pulsing the gate of the transfer transistor504. Typically, the reset transistor 508 is used to reset the voltage onthe sense region 506 (node 516) to a predetermined level prior to thetransfer of charge from the photo detector 502 to the sense region 506.When charge is to be readout of the pixel, the gate of the row selecttransistor is pulsed through the row select 412 and row select line 414to select the pixel (or row of pixels) for readout. The readouttransistor 510 senses the voltage on the sense region 506 and the rowselect transistor 512 transmits the voltage to the output line 410. Theoutput line 410 is connected to readout circuitry and (optionally animage processor) through the output line 410 and the column select 408.

In some embodiments, an image capture device, such as a camera, may notinclude a shutter over the lens, and so the image sensor may beconstantly exposed to light. In these embodiments, the photo detectorsmay have to be reset or depleted before a desired image is to becaptured. Once the charge from the photo detectors has been depleted,the transfer gate and the reset gate are turned off, isolating the photodetectors. The photo detectors can then begin integration and collectingphoto-generated charge.

FIG. 6 illustrates a simplified schematic view of one example of asub-pixel arrangement suitable for use in an image sensor. Four photodetectors 600, 602, 604, 606 are each connected to a separate transfertransistor 608, 610, 612, 614. Each photo detector and associatedtransfer transistor can form one sub-pixel 616. Thus, there are foursub-pixels in the illustrated embodiment. The four sub-pixels 616 canrepresent a pixel in a pixel array in some embodiments. Otherembodiments can have two or more sub-pixels represent a pixel.

The transfer transistors 608, 610, 612, 614 are each connected to acommon node 618. Common node 618 can be implemented as the sense region506 in FIG. 5. The common node is shared by the four sub-pixels 616.

Readout circuitry 620 can be connected to the common node 618. By way ofexample only, a sense region, a reset transistor, and a readouttransistor can be included in the readout circuitry 620 and can beconfigured as shown in FIG. 5. The sense region, the reset transistorand the readout transistor can be connected to the common node 618. Arow select transistor can be connected to the readout transistor.

The gates of each transfer transistor 608, 610, 612, 614 can beselectively pulsed in one embodiment, allowing charge from one or morephoto detectors to transfer to the common node 618. Thus, the sub-pixels616 can provide a charge summing mode. Since the transfer transistors608, 610, 612, 614 can each be selectively pulsed, the charge from one,two, three, or four photo detectors can be transferred separately, incombinations, or simultaneously. For example, the charge from photodetectors 600 and 604 can be summed together prior to readout byseparately or simultaneously pulsing the gates of the respectivetransfer transistors 608 and 612, thereby transferring the charge to thecommon node 618. In one embodiment, charge from multiple sub-pixels thatrepresents a single color is summed together (e.g., charge from two redsub-pixels is summed). The summed charge can then be readout using someor all of the components in the readout circuitry 616.

An image sensor that includes the sub-pixels 616 can operate in a highdynamic range mode. For example, in the illustrated embodiment the photodetectors in two sub-pixels can have a first integration time while thephoto detectors in the other two sub-pixels can have a different secondintegration time. When the first integration time is shorter than thesecond, the sub-pixels with the first integration time can bettercapture the brighter areas in a scene and the sub-pixels with the secondintegration time can better capture darker areas in the scene. Thecharge from the two sub-pixels with the first integration time can besummed together by selectively transferring the accumulated charge inthe photo detectors to the common node 618 and then reading the chargeout using some or all of the components in the readout circuitry 616.Likewise, the charge from the two sub-pixels with the second integrationtime can be summed together by selectively transferring the accumulatedcharge in the photo detectors to the common node 618 and then readingout the summed charge. A final high dynamic range image can be obtainedby combining or stitching both images together.

An image sensor can be constructed on a single semiconductor-based waferor on multiple semiconductor-based wafers. When a single wafer is used,the components in each pixel reside in or on the single wafer. Whenmultiple wafers are used, the components in each pixel can be dividedbetween two or more wafers. For example, in the embodiment illustratedin FIG. 5, the photo detectors and the transfer transistors can resideon one wafer and the sense regions, reset transistors, readouttransistors and row select transistors on a different wafer.Alternatively, with the FIG. 6 embodiment, the photo detectors and thetransfer transistors can reside on a first wafer and the common senseregion on a second wafer. The reset, readout, and row select transistorscan also be formed in or on the second wafer and can be shared by two ormore photo detectors on the first wafer. An interconnect layer istypically used to electrically connect the transfer transistors to thesense region or regions.

In general, photo detectors detect light with little or no wavelengthspecificity, making it difficult to identify or separate colors. Whencolor separation is desired, a color filter array can be disposed overthe imaging area to filter the wavelengths of light sensed by the photodetectors in the imaging area. A color filter array is a mosaic of colorfilters with each color filter typically disposed over a respectivepixel. Each color filter restricts the wavelengths of light detected bythe photo detector, which permits color information in a captured imageto be separated and identified.

FIG. 7 depicts one color filter array suitable for use with an imagesensor having pixels configured as shown in FIG. 5. The color filterarray (CFA) 700 includes filter elements 702, 704, 706, 708. Althoughonly a limited number of color filters are shown, those skilled in theart will recognize that a color filter can include thousands or millionsof color filters.

In one embodiment, each filter element restricts light wavelengths. Inanother embodiment, some of the filter elements filter light wavelengthswhile other filter elements are panchromatic. A panchromatic colorfilter can have a wider spectral sensitivity than the spectralsensitivities of the other color filters in the CFA. For example, apanchromatic filter can have a high sensitivity across the entirevisible spectrum. A panchromatic filter can be implemented, for example,as a neutral density filter or a color filter. Panchromatic filters canbe suitable in low level lighting conditions, where the low levellighting conditions can be the result of low scene lighting, shortexposure time, small aperture, or other situations where light isrestricted from reaching the image sensor.

Color filter arrays can be configured in a number of different mosaics.The color filter array 700 can be implemented as a red (R), green (G),and blue (B) color filter array or a cyan (C), magenta (M), yellow (Y)color filter array. The Bayer pattern is a well know color filter arraypattern. The Bayer color filter array filters light in the red (R),green (G), and blue (B) wavelengths ranges (see, e.g., FIG. 8). TheBayer color filter pattern includes two green color filters (Gr and Gb),one red color filter, and one blue color filter. The group of four colorfilters is tiled or repeated over the pixels in an imaging area to formthe color filter array.

FIG. 9 illustrates a color filter array suitable for use with an imagesensor having sub-pixels configured as shown in FIG. 6. Each colorfilter is divided into four sub-color filters C1, C2, C3, C4, and eachsub-color filter is disposed over a photo detector in a sub-pixel. Thus,in the FIG. 6 embodiment, the four sub-pixels detect light associatedwith the same color. The charge in two or more photo detectors can besummed together, thereby allowing charge representing the same color tobe summed. For example, the charge in the photo detectors covered byfilter elements C1 and C4 can be summed together, and the charge in thephoto detectors covered by filter elements C2 and C3 can be summedtogether.

As described earlier, the sub-pixels can permit an image sensor to beconfigured with different operating modes. One mode is a charge summingmode. Another mode is a high dynamic range mode. Turning to theembodiment illustrated in FIG. 9, a high dynamic range mode can providea first integration period to the photo detectors associated with filterelements C1 and C4. The photo detectors associated with filter elementsC2 and C3 can have a second different integration time. Because thephoto detectors with the first integration time are diagonally opposedto each other, and the photo detectors with the second integration timeare diagonally opposed to each other, the images captured by the fourphoto detectors can have the same center of gravity. Additionally, thespatial resolution in both the horizontal and vertical directions can bethe same in the final high dynamic range image, thereby reducing oreliminating spatial artifacts.

Referring now to FIG. 10, there is shown a top view of an asymmetricalphoto detector pair. The pair 1000 includes two adjacent sub-pixels inone embodiment. The sub-pixels can be vertically or horizontallyadjacent. Portions 1002 of the photo detectors are covered by a lightshield (not shown) and portions 1004 of the photo detectors are notcovered by the light shield. The portions 1004 of the photo detectorsthat are not covered by the light shield are able to detect light, sothe light shield may only partially block incident light on the photodetectors.

FIG. 11 illustrates a simplified cross-sectional view of theasymmetrical photo detector pair shown in FIG. 10. One sub-pixel 1100includes a photo detector 1102 formed in a substrate 1104. The substratecan be a semiconductor-based material similar to those described inconjunction with the support structure 306 in FIG. 3. Another sub-pixel1106 includes a photo detector 1108 formed in the substrate 1104. Alight shield 1110 is disposed over portions of both of the photodetectors 1102, 1108. The light shield 1110 can be made of any opaquematerial or combination of materials, including, but not limited to, ametal.

In the illustrated embodiment, the light shield 1110 is disposed overcontiguous portions of the photo detectors. Other embodiments are notlimited to this construction. The light shield 1110 can cover differentnon-contiguous portions of the photo detectors. The light shield can bedisposed over any given portion of the photo detectors. For example, thelight shield can be disposed over half of each photo detector.

Additionally, each asymmetrical photo detector pair in an imaging areacan be identical or some of the asymmetrical photo detector pairs candiffer from other asymmetrical photo detector pairs in an imaging area.By way of example only, the light shield in some of the asymmetricalphoto detector pairs can be disposed over half of the photo detectorswhile the light shield in other asymmetrical photo detector pairs cancover a third of the photo detectors.

Filter elements 1112, 1114 are disposed over the photo detectors 1102,1108, respectively. The filter elements 1112, 1114 can filter lightwavelengths that represent the same or different colors. A microlens1116, 1118 is disposed over each filter element 1112, 1114. Themicrolenses 1116, 1118 are configured to focus incident light 1120 ontorespective photo detectors 1102, 1108. The light 1120 is angled from theleft in the illustrated embodiment. The light shield 1110 blocks some orall of the light 1120 received by the sub-pixel 1100, thereby preventingthe photo detector 1102 from detecting all of the light that would beincident on the photo detector 1102 if the light shield were notpresent. Similarly, the light shield 1110 blocks some or all of thelight 1120 received by the sub-pixel 1102, thereby preventing the photodetector 1108 from detecting all of the light that would be incident onthe photo detector 1108 if the light shield were not present. Due to thedirection and angle of the light 1120 and the light shield 1110, thephoto detector 1108 can detect more light 1120 than the photo detector1102. Thus, the photo detector 1108 can accumulate more charge than thephoto detector 1102, making the signal response of the sub-pixel 1106higher than the signal response of the sub-pixel 1100.

When the light 1120 is angled from the right (not shown), the signalresponse of the sub-pixel 1100 can be higher than the signal response ofthe sub-pixel 1106. And when the light is perpendicular to the surfaceof the substrate 1104, partial light will be blocked on both sub-pixels1100, 1106, and the signal responses of both sub-pixels 1100, 1106 canbe substantially the same. Object phase information can be obtained byanalyzing the signal responses of the sub-pixels in the asymmetricalpair. The object phase information can be used to provide informationabout the field depth.

FIG. 12 illustrates an example of angular signal responses of theasymmetrical photo detector pair shown in FIG. 11. Since the lightshield 1110 in the FIG. 11 embodiment is symmetrical and the lightshield is disposed over half of each photo detector 1102, 1108, the twosignal responses are both shaped and are reversed with respect to eachother (i.e., mirror images), and the two signal responses intersect atthe point 1204 when the angle of incident light is substantially zero.Other embodiments can shape the signal responses differently.

A focus setting can be determined for an image or images by analyzingthe differences between the signal responses produced by theasymmetrical photo detector pair. For example, the lens can bepositioned or moved to find the optimum focal spot for the remainingsub-pixels in a pixel or for a pixel. When a lens is not properlyfocused, the signal levels obtained from each sub-pixel can be plottedon a respective signal response. The position of the lens can then bechanged to cause the signal levels plotted on the signal responses to belocated at the intersection point 1204. For example, points 1200, 1202represent signal response levels obtained from the sub-pixels 1100,1106. The lens of an image capture device can be adjusted so that thesignal response levels obtained from the sub-pixels move on or near theintersection point 1204.

Referring now to FIG. 13, there is shown a simplified top view of animaging area showing one arrangement of asymmetrical photo detectorpairs. The imaging area 1300 includes pixels 1302 that are each dividedinto four sub-pixels 1304. The pixels are arranged in rows and columns,and in one embodiment one or more rows can include asymmetrical photodetector pairs 1306. In another embodiment, one or more columns caninclude the asymmetrical photo detector pairs.

As described earlier, since each photo detector in a sub-pixel can beconnected to a transfer gate, the asymmetrical photo detector pairs 1306can be read out separately from the other sub-pixels 1304 in a pixel1302. When the imaging area 1300 will be used to capture a high dynamicrange image, the charge produced from the other sub-pixels in a pixelthat includes an asymmetrical photo detector pair is read out and usedto produce the final high dynamic range image. One of the sub-pixelsreadout can have a first integration time and the other sub-pixel canhave a different second integration time. Since the locations of theasymmetrical photo detector pairs are known, the final high dynamicrange image can be obtained by scaling (e.g., multiply by 2) the chargeor image signal readout from the two sub-pixels in the pixels havingasymmetrical photo detector pairs. By way of example only, the scalingcan be performed in the digital domain with the imaging domain for imagereconstruction.

One or more embodiments can include the asymmetrical photo detectorpairs in an entire row or column of an imaging area or in selectedlocations in the imaging area. An imaging area can include any number ofrows or columns or select locations of asymmetrical photo detectorpairs. By way of example only, ten or twenty rows or columns can includeasymmetrical photo detector pairs.

FIG. 14 illustrates a simplified top view of an image sensor showinganother arrangement of asymmetrical photo detector pairs. Theasymmetrical photo detector pairs 1400 are positioned at selectlocations in an imaging area 1402. The charge in the asymmetrical photodetector pairs can be readout separately from the other sub-pixels whendepth information is to be determined. The charge in the asymmetricalphoto detector pairs may not be read out when a high dynamic range imageis to be produced.

Referring now to FIG. 15, there is shown a flowchart of a method fordetermining an initial focus setting for an image capture device.Initially, an image is captured by the pixels or sub-pixels in animaging area using a focus setting of N. N can be any given focussetting. The charge in one or more asymmetrical photo detector pairs isreadout and the signal responses are analyzed (block 1502). The signalresponses can be analyzed, for example, as described in conjunction withFIG. 12.

A determination is then made at block 1504 as to whether a differencebetween the signal responses output from at least one asymmetrical photodetector pair is equal to or less than a threshold value. For example,the threshold value can be a range or a percentage that defines anacceptable difference between the two signal responses. If thedifference between the signal responses is equal to or less than thethreshold value, an initial focus setting is set to setting N (block1506). The initial focus setting can be used to capture one or moreimages.

If however, the difference between the signal responses is greater thanthe threshold value, focus setting N is changed to another setting valueN=N+M (block 1508). M can be any given number, thereby allowing thefocus setting to increase or decrease sequentially or by a certainnumber of settings. The change in the focus setting (increase ordecrease) can be based on the analysis of the signal responses and thedirections of change needed to bring the signal responses near or at theintersection point. The method then returns to block 1502 and repeatsuntil the difference in the signal responses is equal or less than thethreshold value.

The process illustrated in FIG. 15 can be performed at any time. Forexample, a camera can perform the method when the camera is firstpowered on. Additionally or alternatively, a camera can perform theprocesses at predetermined times or at selected times. For example, auser can interact with an input device, such as an icon on a displayscreen, to instruct the camera to perform the method.

The embodiments described herein can provide an image sensor or imagecapture device that is capable of operating in three different modes; acharge summing mode, a high dynamic range mode, and a depth of fieldinformation mode. The depth of field information can be obtained usingone or more asymmetrical photo detector pairs included in an imagingarea of an image sensor. Each asymmetrical photo detector pair includestwo photo detectors and a light shield disposed over a portion of thetwo photo detectors to partially block light that is incident on the twophoto detectors. The signal responses produced by the two photodetectors in one or more asymmetrical photo detector pairs can beanalyzed to determine a focus setting for an image capture device. Acolor filter array can be disposed over the imaging area. The filterelements disposed over the two photo detectors in each asymmetricalphoto detector pair can filter light representing the same color ordifferent colors.

Various embodiments have been described in detail with particularreference to certain features thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the disclosure. For example, the embodiments described herein havedescribed the asymmetrical photo detector pairs as being constructed ina sub-pixel. Other embodiments can locate the asymmetrical photodetector pairs in pixels in an imaging area, where each pixel includes aphoto detector and a transfer gate. FIG. 5 illustrates an example ofsuch pixels.

And even though specific embodiments have been described herein, itshould be noted that the application is not limited to theseembodiments. In particular, any features described with respect to oneembodiment may also be used in other embodiments, where compatible.Likewise, the features of the different embodiments may be exchanged,where compatible.

We claim:
 1. An image capture system, comprising: an image sensorcomprising a plurality of pixels, at least one pixel comprising: a firstsub-pixel comprising a first photodetector; a second sub-pixel adjacentthe first sub-pixel, the second sub-pixel comprising a secondphotodetector separated from the first photodetector by a boundary; athird sub-pixel comprising a third photodetector; and a fourth sub-pixelcomprising a fourth photodetector; a light shield disposed above theimaging area and positioned to overlap the boundary and a portion of thefirst photodetector and the second photodetector; and a processor incommunication with the imaging area and configured to: receive a firstsignal from the first photodetector and a second signal from the secondphotodetector in response to a first focus setting of the image sensor;change the first focus setting to a second focus setting upondetermining that a difference between the first signal and the secondsignal is greater than a threshold value; receive a third signal fromthe third photodetector and a fourth signal from the fourthphotodetector, the third and the fourth signals obtained during an imagecapture; and scale the third signal and the fourth signal to produce animage obtained during the image capture.
 2. The image capture system ofclaim 1, wherein: the third signal is obtained using a first integrationtime; and the fourth signal is obtained using a second integration time.3. The image capture system of claim 1, wherein the third signal and thefourth signal are scaled by a same value.
 4. The image capture system ofclaim 1, wherein: the third signal is scaled by a first value; and thefourth signal is scaled by a second value.
 5. The image capture systemof claim 1, wherein the portion of the first photodetector and thesecond photodetector overlapped by the light shield comprises half of anarea of each of the first photodetector and the second photodetector. 6.The image capture system of claim 1, further comprising: a firsttransfer transistor operably connected to the first photodetector; asecond transfer transistor operably connected to the secondphotodetector; a third transfer transistor operably connected to thethird photodetector; and a fourth transfer transistor operably connectedto the fourth photodetector.
 7. The image capture system of claim 6,wherein the third and the fourth transfer transistors are connected to acommon node.
 8. The image capture system of claim 1, further comprisinga color filter array disposed over the imaging area, wherein a filterelement in the color filter array is disposed over the first, thesecond, the third, and the fourth photodetectors.
 9. The image capturesystem of claim 8, wherein the filter element filters light representinga single color.
 10. An electronic device, comprising: an image sensorcomprising: an asymmetric pixel pair comprising: a first photodetectoradjacent to a second photodetector; and a light shield layer positionedto overlap a portion of the first photodetector and the secondphotodetector; a first pixel comprising a third photodetector and afirst transfer transistor operably connected to the first photodetector;and a second pixel comprising a fourth photodetector and a secondtransfer transistor operably connected to the second photodetector,wherein the first and the second photodetectors are operably connectedto a common node; and a processor in communication with the imaging areaand configured to: obtain a first signal from the first photodetectorand a second signal from the second photodetector based on a first focussetting of the image sensor; change the first focus setting to a secondfocus setting upon determining that a difference between the firstsignal and the second signal is greater than a threshold value; obtain athird signal from the third photodetector after a first integrationtime; and receive a fourth signal from the fourth photodetector after adifferent second integration time; scale the third signal and the fourthsignal; and generate an image using the scaled third and fourth signals.11. The electronic device of claim 10, wherein the processor generatesthe image by: causing charge in the third photodetector to betransferred to the common node using the first transfer transistor;causing charge in the fourth photodetector to be transferred to thecommon node using the second transfer transistor; and causing a summedsignal to be read from the common node.
 12. The electronic device ofclaim 10, wherein the light shield layer is disposed over neighboringportions of the first and the second photodetectors.
 13. The electronicdevice of claim 12, wherein the neighboring portions of the first andthe second photodetectors comprise at least half of a surface area ofthe first and the second photodetectors.
 14. The electronic device ofclaim 10, further comprising a color filter array that includes a firstcolor filter element disposed over the third photodetector and a secondcolor filter element disposed over the fourth photodetector, wherein thefirst and the second color filter elements filter light representing asame color.
 15. A method for operating an image sensor having anasymmetrical pixel pair, a first imaging pixel, and a second imagingpixel, the method comprising: capturing an image using a first focussetting; analyzing a first signal received from a first photodetector inthe asymmetrical pixel pair; analyzing a second signal received from asecond photodetector in the asymmetrical pixel pair; determining if adifference between the first and the second signals is equal to or lessthan a threshold value; if the difference between the first and thesecond signals is greater than the threshold value, changing the firstfocus setting to a second focus setting; and capturing a second image,comprising: receiving a third signal from the first imaging pixel,wherein the third signal is based on a first integration time of thefirst imaging pixel; receiving a fourth signal from the second imagingpixel, wherein the fourth signal is based on a different secondintegration time; and generating the second image from the third signaland the fourth signal.
 16. The method of claim 15, wherein: the thirdsignal is scaled using a first value; and the fourth signal is scaledusing a second value.
 17. The method of claim 15, wherein the thirdsignal and the fourth signal are scaled by a same value.
 18. The methodof claim 15, wherein: receiving the third signal comprises transferringcharge from the first imaging pixel to a common node using a firsttransfer transistor; receiving the fourth signal comprises transferringcharge from the second imaging pixel to the common node using a secondtransfer transistor; and generating the second image comprises reading asummed signal from the common node.
 19. The method of claim 15, whereinthe second image is captured using the second focus setting when thedifference between the first and the second signals is greater than thethreshold value; and the second image is captured using the first focussetting when the difference between the first and the second signals isless than the threshold value.
 20. The method of claim 15, furthercomprising prior to capturing the second image: repeatedly capturing oneor more images using a different focus setting; analyzing the first andthe second signals; and determining if a difference between the firstand the second signals is equal to or less than a threshold value untilthe difference between the first and the second signals is equal to orless than the threshold value, wherein the second focus setting is setto a focus setting associated with the difference between the first andthe second signals being equal to or less than the threshold value.